Privacy-Preserved Distributed Learning With Zeroth-Order Optimization

We develop a privacy-preserving distributed algorithm to minimize regularized empirical risk function when the first-order information is not available and data over multi-agent network. employ zeroth-order method associated augmented Lagrangian in primal domain using alternating direction of multipliers (ADMM). show that proposed algorithm, named ADMM (D-ZOA), has intrinsic properties. Most existing optimization/estimation algorithms exploit some perturbation mechanism preserve privacy, which comes at cost reduced accuracy. Contrarily, by analyzing inherent randomness due use method, we D-ZOA intrinsically endowed with $(\epsilon,\delta)-$ differential privacy. In addition, moments accountant total privacy leakage grows sublinearly number iterations. outperforms differentially-private approaches terms accuracy while yielding similar guarantee. prove reaches neighborhood optimal solution whose size depends on parameter. The convergence analysis also reveals practically important trade-off between Simulation results verify desirable properties its superiority state-of-the-art as well network-wide convergence.